It looks like existing conf files need to be updated; thepandagma pan GitHub Action workflow jobs are failing with errors like:

2024-02-28T11:28:44.5818665Z /home/runner/work/pandagma/pandagma/bin/pandagma-common.sh: line 81: min_align_count: unbound variable
2024-02-28T11:28:44.5838371Z ERROR conda.cli.main_run:execute(124): `conda run pandagma pan -d data_TEfilter -c config/Arachis3_5_2_0.conf -w /work -r -o upload-artifact/pandagma-pan-Arachis3_5_2_0` failed. (See above for error)

https://github.com/legumeinfo/pandagma/actions/runs/8078741346

I think I've fixed at least those config omissions.

From the results of the last workflow run:
https://github.com/legumeinfo/pandagma/actions/runs/8087305867

The new default workflow steps in pandagma pan (align_cds, align_protein, model_and_trim, calc_trees xfr_aligns_trees) resulted in a >3X increase in runtime, causing the Medicago3_8_0_15 to exceed the GitHub Actions 6-hour job walltime limit. I'll remove that from the default list of configurations that are tested.

The Glycine_7_3_2 and Vigna4_7_1_4 jobs completed, but encountered an OS "too many open files" error when running actions/upload-artifact; e.g.:

With the provided path, there will be 152583 files uploaded
Artifact name is valid!
Root directory input is valid!
Beginning upload of artifact content to blob storage
node:events:492
      throw er; // Unhandled 'error' event
      ^

Error: EMFILE: too many open files, open '/home/runner/work/pandagma/pandagma/upload-artifact/pandagma-pan-Glycine_7_3_2/22_hmmalign/pan22656'
Emitted 'error' event on ReadStream instance at:
    at emitErrorNT (node:internal/streams/destroy:151:8)
    at emitErrorCloseNT (node:internal/streams/destroy:[11](https://github.com/legumeinfo/pandagma/actions/runs/8087305867/job/22098949476#step:8:12)6:3)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
  errno: -24,
  code: 'EMFILE',
  syscall: 'open',
  path: '/home/runner/work/pandagma/pandagma/upload-artifact/pandagma-pan-Glycine_7_3_2/22_hmmalign/pan22656'
}

Node.js v[20](https://github.com/legumeinfo/pandagma/actions/runs/8087305867/job/22098949476#step:8:21).8.1

The previous successful run of Glycine_7_3_2 resulted in only 23 files in the out_pandagma directory (instead of 152583 files).

The extra files are in the 21_hmm, 22_hmmalign, 23_hmmalign_trim2, and 24_trees directories. Are these needed in the out_pandagma for pandagma pan, or only in work_pandagma?

For the default workflows, I think we should NOT include these steps:

pan: align_cds, align_protein, model_and_trim, calc_trees, xfr_aligns_trees
fam: align_protein, model_and_trim, calc_trees, xfr_aligns_trees

Omitting these should fix both the runtime and file-count problems.

What determines which steps are included in "default" in the GitHub actions?

What determines which steps are included in "default" in the GitHub actions?

The GitHub Actions workflow is running the default sequence of pandagma workflow steps run when the pandagma pan -s <STEP> option is omitted:

The GitHub Actions workflow is running the default sequence of pandagma workflow steps run when the pandagma pan -s option is omitted

OK. I'll pull the optional steps out of -pan and -fam (just removing them from commandlist) and do some more testing. I'll convert this PR to draft until then.

@nathanweeks Ready for review again. I tested the fam workflow with family_7_3.conf, the pan workflow with Vigna4_7_1_4.conf, and the fsup workflow with family3_sup.conf and family3_sup_chafa.conf (but I would say don't bother testing fsup; I think I've got that one - and it is still peripheral.)

GA workflow passed.